High performance cloud-scale networking applications process millions of packets a second. These cloud-scale applications are required for complex and resource-intensive processes, such as spinning up new virtual machines (VMs) and sharing an existing load over those VMs, shutting down VMs and redistributing their load across existing VMs, and dealing with the unexpected shutdown of VMs due to software fault or failure of the machine on which they are running. In view of these challenges, there is a well-established design pattern used with cloud-scale web applications, where a load balancer and some form of distributed datastore are used. This functionality typically operates independently of the cloud-scale web application itself.
However, high performance networking applications face a different challenge with respect to web applications, in that they can implement a so-called “bump-in-the-wire” approach on high-speed network traffic, where, for example, a transparent firewall resides between groups of hosts and has the ability to filter traffic between hosts within the subnet. In such a case, data packets must be continuously processed, and outages as short as 50 ms may be unacceptable. For example, in a network of VMs, when a VM is lost, data will likely need to be re-fetched and sent to the VM that receives the flow of the lost VM as a result of rebalancing. Additionally, should a VM be added to or removed from the network through scaling, data will need to be redistributed in order to rebalance the flows, even though packets cannot be processed until the data has arrived. In either case, a delay is likely to occur as data is recovered on the new machine from the location at which the data backup exists. As such, when redistributing the VM network flows due to one of the above events, it is important that the flows be rebalanced to VMs that already contain a copy of the state of the flow, without having to first replicate all data on all nodes, which can take too long and require too much network traffic to be feasible.